Information processing device, information processing method, and program

ABSTRACT

Provided is an information processing apparatus including a sound output control unit configured to generate localization information of a sound marker based on a virtual position, and a sound output unit configured to output a sound associated with the sound marker, based on the localization information, wherein the virtual position is determined based on a position of a real object present in a space.

TECHNICAL FIELD

The present disclosure relates to an information processing device, aninformation processing method, and a program.

BACKGROUND ART

In recent years, the amount of information that users obtain hascontinued to increase. For at least this reason, questions of howinformation necessary for users will be extracted, or how informationwill be provided, have become increasingly important. Thus, presentationof information using sounds has been proposed. PTL 1, for example,discloses an information processing device that provides users withinformation using sounds having directivity. A user can select a soundusing the orientation of his or her face from a plurality of soundsprovided by the information processing device.

CITATION LIST Patent Literature PTL 1: JP 2008-92193A

Attention has also been directed to a technology called augmentedreality (AR) in which additional information is presented to users bybeing superimposed on a real image. In the AR technology, informationpresented to users is also called an annotation, which can be visualizedas a virtual object in various forms of, for example, text, icons, andanimations.

SUMMARY Technical Problem

Presentation of information using sounds has been seldom discussed inthe field of AR technology. Therefore, the present disclosure proposes anovel and improved information processing device, information processingmethod, and program which are capable of generating localizationinformation of a sound pertaining to a virtual object based on aposition of the virtual object.

Solution to Problem

According to an embodiment of the present disclosure, there is providedan information processing apparatus including a sound output controlunit configured to generate localization information of a sound markerbased on a virtual position, and a sound output unit configured tooutput a sound associated with the sound marker, based on thelocalization information, wherein the virtual position is determinedbased on a position of a real object present in a space.

Further, according to an embodiment of the present disclosure, there isprovided an information processing method including generatinglocalization information of a sound marker based on a virtual position,and outputting a sound associated with the sound marker, based on thelocalization information, wherein the virtual position is determinedbased on a position of a real object present in a space.

Further, according to an embodiment of the present disclosure, there isprovided a non-transitory computer-readable medium embodied with aprogram, which when executed by a computer, causes the computer toperform a method including generating localization information of asound marker based on a virtual position, and outputting a soundassociated with the sound marker, based on the localization information,wherein the virtual position is determined based on a position of a realobject present in a space.

Further, according to an embodiment of the present disclosure, there isprovided an information processing device including an acquisition unitthat acquires a virtual position of a virtual object determined based ona position of a real object present in a space, and a localizationinformation generation unit that generates localization informationindicating a sound source position of sound information associated withthe virtual object based on the virtual position.

Further, according to an embodiment of the present disclosure, there isprovided an information processing method including acquiring a virtualposition of a virtual object determined based on a position of a realobject present in a space, and generating localization informationindicating a sound source position of sound information associated withthe virtual object based on the virtual position.

Further, according to an embodiment of the present disclosure, there isprovided a program that causes a computer to function as an informationprocessing device including an acquisition unit that acquires a virtualposition of a virtual object determined based on a position of a realobject present in a space, and a localization information generationunit that generates localization information indicating a sound sourceposition of sound information associated with the virtual object basedon the virtual position.

Advantageous Effects of Invention

According to the present disclosure as described above, it is possibleto generate localization information of a sound pertaining to a virtualobject based on a position of the virtual object.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is an illustrative diagram showing the overview of userexperience provided by an information processing device according to anembodiment of the present disclosure.

FIG. 2 is an illustrative diagram showing an example of a visual markerprovided by the information processing device according to an embodimentof the present disclosure.

FIG. 3 is an illustrative diagram showing another example of the visualmarker provided by the information processing device according to anembodiment of the present disclosure.

FIG. 4 is an illustrative diagram showing an example of an operation forthe information processing device according to an embodiment of thepresent disclosure.

FIG. 5 is a block diagram showing a function configuration of theinformation processing device according to an embodiment of the presentdisclosure.

FIG. 6 is a block diagram showing a hardware configuration example ofthe information processing device according to an embodiment of thepresent disclosure.

FIG. 7 is a flowchart for describing an example of an informationpresentation operation of the information processing device according toan embodiment of the present disclosure.

FIG. 8 is a flowchart for describing an example of a presentationoperation of second information and a sound in the informationprocessing device according to an embodiment of the present disclosure.

FIG. 9 is an illustrative diagram showing an example of generatinglocalization information when information is passed over without auser's operation of selecting primary information in the informationprocessing device according to an embodiment of the present disclosure.

FIG. 10 is an illustrative diagram showing an example of generatinglocalization information when the user performs an operation ofselecting primary information in the information processing deviceaccording to an embodiment of the present disclosure.

FIG. 11 is an illustrative diagram showing an example of generatinglocalization information in a first state when the user changes theorientation of his or her face in the information processing deviceaccording to an embodiment of the present disclosure.

FIG. 12 is an illustrative diagram showing an example of generatinglocalization information in a second state when the user changes theorientation of his or her face in the information processing deviceaccording to an embodiment of the present disclosure.

FIG. 13 is an illustrative diagram showing an example of generatinglocalization information in a third state when the user changes theorientation of his or her face in the information processing deviceaccording to an embodiment of the present disclosure.

DESCRIPTION OF EMBODIMENTS

Hereinafter, embodiments of the present disclosure will be described indetail with reference to the appended drawings. Note that, in thisspecification and the appended drawings, structural elements that havesubstantially the same function and structure are denoted with the samereference numerals, and repeated explanation of these structuralelements is omitted.

Note that description will be provided in the following order.

1. Overview

2. Configuration Example

2-1. Function Configuration Example

2-2. Hardware Configuration Example

3. Operation Example

4. Generation of Localization Information

1. Overview

First, the overview of user experience provided by an informationprocessing device according to an embodiment of the present disclosurewill be described with reference to FIGS. 1 to 4. FIG. 1 is anillustrative diagram showing the overview of user experience provided bythe information processing device according to an embodiment of thepresent disclosure. FIG. 2 is an illustrative diagram showing an exampleof a visual marker provided by the information processing deviceaccording to an embodiment of the present disclosure. FIG. 3 is anillustrative diagram showing another example of the visual markerprovided by the information processing device according to an embodimentof the present disclosure. FIG. 4 is an illustrative diagram showing anexample of an operation for the information processing device accordingto an embodiment of the present disclosure.

The information processing device 10 according to an embodiment of thepresent disclosure may provide information associated with a real objectpresent in a real space using visual information and sound information.Hereinafter, visual information will also be referred to as a visualmarker Mv or a virtual object. In addition, sound information will alsobe referred to as a sound marker Ms. The information processing device10 may determine a position of the visual marker Mv based on, forexample, a position of a real object. In addition, the informationprocessing device 10 may determine a sound source position of the soundmarker Ms based on the position of the visual marker Mv. The informationprocessing device 10 may display the visual marker Mv, for example, overthe position of a real object in an imaged video in a superimposingmanner. In addition, the information processing device 10 may determinethe sound source position of the sound marker Ms as if the sound markeris heard from the position of the real object in the imaged video (FIG.1). Note that FIG. 1 is merely an image, and the sound marker Ms mayactually be invisible. The sound marker Ms may be provided using asound. Furthermore, the information processing device 10 may provide auser with information obtained by filtering information associated withreal objects located in the vicinity of the user according to thepreference, the purpose, or the like of the user.

FIG. 2 illustrates an example of the visual marker Mv. The visual markerMv may be provided to a user by being displayed in a superimposingmanner over a video imaged by, for example, a camera. In addition, thevisual marker Mv may be provided using an HMD (Head Mounted Display).Filtered information may be first provided as, for example, primaryinformation Mv1 of the visual marker Mv. The primary information Mv1 maybe, for example, information which is associated with the correspondingposition having granularity as brief as to inform a user of the overviewof the information. When the user does not perform any operation for theprimary information Mv1, the information is passed over.

On the other hand, when the user performs a selection operation for thedisplayed primary information Mv1, further detailed information may beprovided to the user. For example, according to the selection operation,secondary information Mv2 that includes more detailed information thanthe primary information Mv1 may be displayed. In addition, a soundmarker Ms may be provided with the display of the secondary informationMv2. Specifically, the sound marker Ms may be provided as a sound havingdirectivity. The sound source position of the sound marker Ms may bedecided based on the positional relationship between the position of thevisual marker Mv and the position of the user. Accordingly, the user canperceive the position of the visual marker Mv from the direction inwhich the sound marker Ms is heard.

In addition, an example of providing information outside of the visualfield is illustrated in FIG. 3. The information processing device 10 maycause the primary information Mv1 of the visual marker Mv to bedisplayed on the edge of the display region. When a selection operationfor the primary information Mv1 of the information associated with aposition outside the visual field is detected, the informationprocessing device 10 may cause the secondary information Mv2 to bedisplayed on the edge of the display region and may output the soundmarker Ms.

A selection operation for acquiring information using the secondaryinformation Mv2 and the sound marker Ms herein may depend on the kind ofthe information processing device 10. When the information processingdevice 10 is an HMD, for example, the operation may be performed usingthe orientation of the user's face. In addition, when the informationprocessing device 10 is a mobile telephone, the operation may beperformed using the orientation of the user's face in the same manner,or using a touch sensor, a button, or the like. FIG. 4 illustrates anexample of the operation using the orientation of a user's face. Whenthe information processing device 10 detects a nodding movement based onthe orientation of the user's face, for example, the movement can bedetermined to be an operation of acquiring specific information. Inaddition, when the information processing device 10 detects a shakingmovement of the head based on the orientation of the user's face, themovement can be determined to be an operation of cancelling selection ofspecific information.

Note that an example in which shop information associated with a signboard in a street has been illustrated here, but the present technologyis not limited to this example. The present technology may also be usedfor the purpose of, for example, avoiding risks. There are an increasingnumber of people who enjoy jogging or cycling while listening to musicthrough headphones. In such a situation, there is a high possibility ofcausing an accident without being aware of an approaching vehicle sinceit is difficult to hear ambient sounds. Thus, when a vehicle approachesa user, by displaying a virtual engine sound as the sound marker Ms andan image of the vehicle as the visual marker Mv, it may be possible tominimize or avoid risk even while listening to music. Hereinbelow, forrealizing such user experience, an information processing device 10 inaccordance with an embodiment of the present disclosure will bedescribed in detail.

2. Configuration Example

Herein, a configuration of an information processing device 10 accordingto an embodiment of the present disclosure will be described. First, afunction configuration example of the information processing device 10will be described with reference to FIG. 5, and a hardware configurationexample of the information processing device 10 will be described withreference to FIG. 6. FIG. 5 is a block diagram showing the functionconfiguration of the information processing device according to anembodiment of the present disclosure. FIG. 6 is a block diagram showingthe hardware configuration example of the information processing deviceaccording to an embodiment of the present disclosure.

2-1. Function Configuration Example

Referring to FIG. 5, the information processing device 10 according toan embodiment of the present disclosure may include, mainly, anoperation unit 105, an operation information acquisition unit 110, adisplay unit 115, a display control unit 120, a sound output unit 125,and a sound output control unit 130.

(Operation Unit 105)

The operation unit 105 may have a function such as generating an inputsignal for performing an operation that a user desires. The operationunit 105 may include, for example, an input section such as a touchpanel, a mouse, a keyboard, a button, a microphone, a switch, a lever,and various sensors that are used to input information by a user, aninput control circuit that generates and outputs an input signal basedon an input by a user, or the like. For example, the operation unit 105may generate and output an input signal based on the orientation of auser's face detected by a sensor. In addition, the operation unit 105,for example, may detect an operation performed on a touch sensor by auser and then generate and output an input signal based on thecorresponding operation.

(Operation Information Acquisition Unit 110)

The operation information acquisition unit 110 may have a function suchas acquiring operation information input by a user. For example, theoperation information acquisition unit 110 may acquire an input signaloutput from the operation unit 105 as operation information. Theinformation processing device 10 may operate based on the operationinformation acquired by the operation information acquisition unit 110.

(Display Unit 115)

The display unit 115 is an example of an output device, and may be adisplay device such as a liquid crystal display (LCD) device, or anorganic EL (or OLED: Organic Light Emitting Diode) display device. Inaddition, the display unit 115 may be an HMD. The display unit 115 mayoperate according to the control of the display control unit 120.

(Display Control Unit 120)

The display control unit 120 may have a function such as controlling thecontent of a display screen to be provided to a user. For example, thedisplay control unit 120 may cause the visual marker Mv to be displayedon an optical transmission type HMD. Alternatively, the display controlunit 120 may cause a video obtained by superimposing the visual markerMv on a video obtained by imaging the visual field of a user to bedisplayed on a video transmission type HMD. In addition, the displaycontrol unit 120 may control the content of a display screen based onoperation information acquired by the operation information acquisitionunit 110. For example, when a selection operation for the primaryinformation Mv1 of the visual marker Mv is acquired, the display controlunit 120 may cause the secondary information Mv2 corresponding to theprimary information to be displayed on the display screen. Note that thedisplay control unit 120 is an example of an acquisition unit thatacquires a position of a virtual object (hereinafter referred to as avirtual position) determined based on the position of a real objectpresent in a space, but is not limited thereto.

(Sound Output Unit 125)

The sound output unit 125 may have a function such as outputting a soundsignal. The sound output unit 125 may be, for example, a speaker. Inaddition, the sound output unit 125 may be an output terminal thatoutputs a sound signal to a sound output device such as headphones. Thesound output unit 125 may operate according to the control of the soundoutput control unit 130. The sound output device used here may becapable of outputting stereophony according to localization informationof a sound.

(Sound Output Control Unit 130)

The sound output control unit 130 may have a function such ascontrolling the output of sounds to a user. In addition, the soundoutput control unit 130 is an example of a localization informationgeneration unit that generates localization information indicating asound source position, but is not limited thereto. The sound outputcontrol unit 130 may generate localization information indicating asound source position of sound information associated with the same realobject as one relating to a virtual object, for example, one associatedwith a virtual object based on, for example, a virtual position. Thesound output control unit 130 may also generate localization informationbased on the distance between a user position and a virtual position.

Specifically, when the distance between a user position and a virtualposition is equal to or longer than a threshold value that may or maynot be predetermined, the sound output control unit 130 may bring thesound source position closer to the front of the user than to thevirtual position. In addition, when the distance between the userposition and the virtual position is shorter than the threshold value,the sound output control unit 130 may move the sound source positionfrom the front of the user to the direction closer to the virtualposition. Humans tend to have a higher ability to recognize thedirection of a sound from an area of the front than from an area of aside. For this reason, humans can more easily recognize the direction ofa sound at rest, for example, in the front than in the right side. Inaddition, humans tend to more easily recognize the direction of a soundof which the sound source position moves than the direction of a soundat rest. For this reason, when a virtual position is separated from auser position in a predetermined or longer distance, for example, evenwhen the sound source position is located obliquely forward as viewedfrom a user, there is a high possibility that the user is not able torecognize such a difference of sound source positions. Thus, when avirtual position is separated from a user position in a predetermined orlonger distance, the sound source position may be set to be in the frontside of the user, and when the user approaches the virtual position, thesound source position may be moved from the front side of the user to adirection closer to the virtual position, so that the user can recognizethe direction of the virtual position more easily.

In addition, the sound output control unit 130 may change the soundsource position according to changes in the orientation of user's face.In this case, when the position of the virtual object is located at anarea in front of the user that may or may not be predetermined, it maybe desirable to set the sound source position to be the virtualposition. The sound output control unit 130 may output the sound markerMs according to generated localization information.

Hereinabove, examples of the functions of the information processingdevice 10 according to embodiments have been shown. Respectiveconstituent elements described above may be configured usinggeneral-purpose members or circuits or may be configured using hardwarespecialized for the functions of the respective constituent elements. Inaddition, the functions of the respective constituent elements may befulfilled by reading a control program from a storage medium such as aROM (Read Only Memory) or a RAM (Random Access Memory) storing thecontrol program describing procedures in which an arithmetic operationunit such as a CPU (Central Processing Unit) realizes the functions andby interpreting the program. Thus, it is possible to appropriatelychange a configuration to be used according to the level of thetechnology at a time when the present embodiment is implemented.Hereinbelow, an example of a hardware configuration of the informationprocessing device 10 will be shown.

A computer program for realizing each of the functions of theinformation processing device 10 according to embodiments as describedabove may be created and installed in a personal computer, or the like.In addition, a recording medium that stores such a computer program andwhich is readable on the computer may also be provided. The recordingmedium may include, for example, a magnetic disk, an optical disc, amagneto optical disc, a flash memory, and the like. In addition, thecomputer program may be delivered through, for example, the Internet,without using the recording medium.

2-2. Hardware Configuration Example

Next, a hardware configuration example of the information processingdevice 10 according to an embodiment of the present disclosure will bedescribed with reference to FIG. 6. The information processing device 10may include a CPU 150, a RAM 155, a non-volatile memory 160, a displaydevice 165, a face orientation detection device 170, and a sound imagelocalization audio device 175.

(CPU 150)

The CPU 150 may function as, for example, an arithmetic processing unitor a control unit, and may control all or some operations of eachconstituent element based on various programs recorded on the RAM 155,the non-volatile memory 160, or the like.

(RAM 155)

The RAM 155 may temporarily store a program read in the CPU 150, aparameter that is appropriately changed during the execution of theprogram, or the like.

(Non-Volatile Memory 160)

The non-volatile memory 160 may be a storage device that retains storagewithout a power supply. The non-volatile memory 160 may be used as, forexample, an auxiliary storage device or a long-term memory storage. Thenon-volatile memory 160 may be a semiconductor memory, for example, aROM (Read Only Memory), a flash memory, or the like. In addition, thenon-volatile memory 160 may be a magnetic storage device such as an HDD(Hard Disk Drive), an optical disc, or the like.

(Display Device 165)

The display device 165 may be a device, for example, an HMD, or thelike, having a display function. Alternatively, the display device 165may be a display device such as a liquid crystal display (LCD) device,or an organic EL (or OLED: Organic Light Emitting Diode) display device.

(Face Orientation Detection Device 170)

The face orientation detection device 170 may have a function ofdetecting the orientation of the user's face. The face orientationdetection device 170 may be, for example, an HMD having a triaxialacceleration sensor. In this case, the display device 165 and the faceorientation detection device 170 may be regarded as the same device.

(Sound Image Localization Audio Device 175)

The sound image localization audio device 175 may be an audio devicethat can perform sound image localization. The sound image localizationaudio device 175 may output a sound having directivity based onlocalization information.

3. Operation Example

Next, an operation example of the information processing device 10according to an embodiment of the present disclosure will be describedwith reference to FIGS. 7 and 8. FIG. 7 is a flowchart for describing anexample of an information presentation operation of the informationprocessing device according to an embodiment of the present disclosure.FIG. 8 is a flowchart for describing an example of a presentationoperation of second information and a sound in the informationprocessing device according to an embodiment of the present disclosure.

First, referring to FIG. 7, the display control unit 120 may determinewhether information is being presented or not (S100). Then, when it isdetermined that the information is not being presented in Step S100, thedisplay control unit 120 may determine whether the information to bepresented is within a fixed area or not (S105). When it is determinedthat the information to be presented is within the fixed area in thedetermination of Step S105, the display control unit 120 may present theprimary information Mv1 of the visual marker Mv (S110).

On the other hand, when it is determined that the information is beingpresented in step S100, the display control unit 120 may determinewhether the primary information Mv1 is being presented or not (S115).Then, when it is determined that the primary information is not beingpresented in Step S115 (in other words, the secondary information isbeing presented), a determination may be made as to whether theoperation information acquisition unit 110 recognizes head shaking ornot (S140). Then, when head shaking has not been recognized in thedetermination of Step S140, a determination may be made as to whether afixed time has elapsed (from when the secondary information starts to bepresented) or not (Step S145). Then, when a fixed time has elapsed orwhen head shaking has been recognized, the display control unit 120 andthe sound output control unit 130 may finish the presentation ofinformation (S150).

On the other hand, when it is determined that the primary information isbeing presented in the determination of Step S115, it may then bedetermined whether nodding has been recognized by the operationinformation acquisition unit 110 or not (S120). Then, when it isdetermined that nodding has been recognized in the determination of StepS120, the display control unit 120 and the sound output control unit 130may present the user with the secondary information and soundinformation (S125). On the other hand, when nodding has not beenrecognized in the determination of Step S120, a determination may bemade as to whether a fixed time has elapsed (from when the primaryinformation starts to be presented) or not (S130). Then, when it isdetermined that a fixed time has elapsed in the determination of StepS130, the display control unit 120 may finish the presentation of theprimary information (S135).

Now, the presentation operation of the secondary information and thesound information shown in Step S125 will be described in detail withreference to FIG. 8.

Referring to FIG. 8, first, the display control unit 120 may determine adisplay position of the secondary information Mv2 of the visual markerMv (S200). Herein, the secondary information Mv2 may include moredetailed information than the primary information Mv1.

Next, based on the display position of the visual marker Mv,localization information of the sound marker Ms may be generated (S205).The generation of the localization information will be described in moredetail later. Further, the sound source position of the sound marker Msmay be determined based on the position of the visual marker Mv.

Then, the display control unit 120 may display the visual marker Mv, andthe sound output control unit 130 may cause the sound marker Ms to beoutput based on the localization information (S210).

4. Generation of Localization Information

Next, the generation of the localization information will be describedin detail with reference to FIGS. 9 to 13. FIG. 9 is an illustrativediagram showing an example of generating the localization informationwhen information is passed over without a user's operation of selectingthe primary information in an information processing device according toan embodiment of the present disclosure. FIG. 10 is an illustrativediagram showing an example of generating the localization informationwhen the user performs an operation of selecting the primary informationin an information processing device according to an embodiment of thepresent disclosure. FIG. 11 is an illustrative diagram showing anexample of generating the localization information in a first state whenthe user changes the orientation of his or her face in an informationprocessing device according to an embodiment of the present disclosure.FIG. 12 is an illustrative diagram showing an example of generating thelocalization information in a second state when the user changes theorientation of his or her face in an information processing deviceaccording to an embodiment of the present disclosure. FIG. 13 is anillustrative diagram showing an example of generating the localizationinformation in a third state when the user changes the orientation ofhis or her face in an information processing device according to anembodiment of the present disclosure.

As described above, the sound source position of the sound marker Ms maybe determined based on a position of the visual marker Mv. In order fora sound to be heard from the direction of the visual marker Mv, theposition of the visual marker Mv may simply be set at the sound sourceposition of the sound marker Ms. Note that the auditory sense of humanshas various characteristics. Therefore, it may be desirable to determinethe sound source position so that humans easily recognize the directionand distance of a sound in accordance with the characteristic of theauditory sense of humans. Hereinbelow, the sound source position of thesound marker Ms will be discussed along with several given examples.

First, referring to the example of FIG. 9, a case in which a real objectpasses by a side of a user from an oblique front side is considered. Inthis case, since the real object is present within the visible region ofthe user at a time t0, the visual marker Mv may be displayed at theposition of the real object. After that, at a time t1, the real objectis positioned out of the visible region of the user. Thus, at thismoment, the visual marker Mv may be displayed on an edge of the visibleregion (edge of the display screen). Then, the visual marker Mv may bedisplayed on an edge of the visible region until the real object ispositioned at substantially right beside the user.

In this case, the sound source position of the sound marker Ms may belocated closer to the front side of the user than to the position of thevisual marker Mv until the distance between the visual marker Mv and theuser becomes a threshold value that may or may not be predetermined(herein, a virtual radius R having the user as the center). In addition,if the distance between the visual marker Mv and the user is shorterthan the threshold value, the sound source position of the sound markerMs may draw a trace moving on the circumference of the virtual radius Rhaving the user position as the center. For example, the sound sourceposition of the sound marker Ms may rotate on the circumference untilthe position is located right behind the user and then recede.

As described above, humans have high resolution capability for a soundin a region close to the front side. On the other hand, humans have lowresolution capability for a sound in a lateral side. For this reason, itis easier to feel a sense of distance to the sound source position whenthe sound source position gets closer to the user straightly within aregion close to the front side than when the sound source position getscloser to the user obliquely in accordance with the positions of thevisual marker Mv in the example of FIG. 9. In addition, the feeling of asound passing by is more easily recognized by rotating the position ofthe sound along the circumference of the virtual radius R.

Next, referring to FIG. 10, a case in which the user performs aselection operation for the primary information is illustrated. In thiscase, since the real object is present within the visible region of theuser at the time t0, the visual marker Mv may be displayed at theposition of the real object. At this moment, if the user performs aselection operation for the visual marker Mv, the visual marker Mv maymove so as to be displayed in the front side of the user at the times t1and t2. After that, at a time t3, the visual marker Mv may disappear tothe outside of the visible region.

In this case, the sound source position of the sound marker Ms may be onthe front side of the user rather than at the position of the visualmarker Mv at the time t0. Then, the sound source position of the soundmarker Ms may get closer to the user straightly until the distancebetween the user and the visual marker Mv becomes a radius R that may ormay not be predetermined. After that, as the visual marker Mv isdisplayed on the front side of the user, the sound source position ofthe sound marker Ms may move to the front side of the user one time.After that, as the visual marker Mv disappears to the outside of thevisible region, the sound source position of the sound marker Ms mayrecede while rotating on the circumference of the radius R having theuser as the center.

Next, referring to FIGS. 11 to 13, the sound source position of thesound marker Ms when the orientation of the user's face changes will bediscussed. It is assumed that, for example, a real object (t0) ispositioned in the front oblique to the right side. Then, as shown inFIG. 11, the sound source position of the sound marker Ms may be locatedon the front side of the user rather than on the side of the realobject. It is assumed that the orientation of the user's face is changedto the right side in this state. Then, at this moment (t1), the soundsource position of the sound marker Ms may be changed so as to getcloser to the real object, as shown in FIG. 12. In addition, if theorientation of the user's face is further changed until the orientationis set to be the front side of the user, the sound source position ofthe sound marker Ms may be at the position of the real object.

Hereinabove, embodiments of the present disclosure have been describedin detail with reference to accompanying drawings, but the technicalscope of the present disclosure is not limited to the above-describedembodiments. It should be understood by those skilled in the art thatvarious modifications, combinations, sub-combinations and alterationsmay occur depending on design requirements and other factors insofar asthey are within the scope of the appended claims or the equivalentsthereof, and they are of course understood so as to belong to thetechnical scope of the present disclosure.

In the above-described embodiments, for example, the secondaryinformation is configured to be presented when an operation for theprimary information is detected, but the present technology is notlimited to such. For example, detailed information may be presented fromthe beginning. Alternatively, information may be presented in a furthernumber of stages.

Furthermore, in the present specification, the steps described in theflowcharts include not only processes performed in a time series manneralong the described order but also processes executed in parallel or inan individual manner, although not necessarily performed in a timeseries. In addition, it is needless to say that even in steps processedin a time series manner, the order can be appropriately changed withoutchanging an intended result.

Additionally, the present technology may also be configured as below.

(1) An information processing apparatus including:

a sound output control unit configured to generate localizationinformation of a sound marker based on a virtual position: and

a sound output unit configured to output a sound associated with thesound marker, based on the localization information,

wherein the virtual position is determined based on a position of a realobject present in a space.

(2) The information processing apparatus of (1), wherein the soundoutput unit is further configured to audibly output the soundconcurrently with a displaying of a visual marker on a display unit atthe virtual position.(3) The information processing apparatus of (1), wherein the soundoutput control unit is further configured to determine a sound sourceposition of the sound marker.(4) The information processing apparatus of (3), wherein the soundoutput control unit is further configured to determine the sound sourceposition of the sound marker based on the virtual position.(5) The information processing apparatus of (3), wherein the soundoutput control unit is further configured to determine the sound sourceposition of the sound marker based on a distance between a position of auser and the virtual position.(6) The information processing apparatus of (5), wherein when thedistance between the position of the user and the virtual position isgreater than a threshold value, the sound output control unit is furtherconfigured to determine the sound source position of the sound marker tobe at a location that is located closer to being directly in front ofthe user than to the virtual position.(7) The information processing apparatus of (6), wherein when thedistance between the position of the user and the virtual position isless than a threshold value and reducing, the sound output control unitis further configured to determine the sound source position of thesound marker as correspondingly moving from a front side of the usertowards the real object.(8) The information processing apparatus of (5), wherein when thedistance between the position of the user and the virtual position isless than a threshold value, the sound output control unit is furtherconfigured to determine the sound source position of the sound marker tobe located at a corresponding point along a circumference having aradius of the threshold value and centered on the position of the user.(9) The information processing apparatus of (8), wherein as the realobject passes from a front side of the user to a back side of the user,the sound output control unit is further configured to determine thesound source position of the sound marker as moving along thecircumference from the front side of the user to the back side of theuser until reaching a position located directly behind the user at whichpoint the sound source position of the sound marker then recedes awayfrom the user along a path of a vector.(10) The information processing apparatus of (5), wherein:when the distance between the position of the user and the virtualposition is greater than a threshold value, the sound output controlunit is further configured to determine the sound source position of thesound marker to be on a front side of the user, andwhen the user approaches a physical location corresponding to thevirtual position, the sound output control unit is further configured todetermine the sound source position of the sound marker as moving fromthe front side of the user in a direction towards the virtual position.(11) The information processing apparatus of (5), wherein:as the distance between the position of the user and the virtualposition decreases to approach a threshold value, the sound outputcontrol unit is further configured to determine the sound sourceposition of the sound marker as moving closer to the position of theuser along a vector that is parallel to a straight ahead direction ofthe user;as the distance between the position of the user and the virtualposition decreases from the threshold value to a point in time at whichthe virtual position is directly in front of the user, the sound outputcontrol unit is further configured to determine the sound sourceposition of the sound marker a moving along a circumference having aradius of the threshold value and centered on the position of the usertowards a position located directly in front of the user, andas the virtual position moves from being directly in front of the usertowards a back side of the user, the sound output control unit isfurther configured to determine the sound source position of the soundmarker as moving from the position located directly in front of the userand along the circumference towards the back side of the user and out ofa visible region of the user.(12) The information processing apparatus of (5), wherein the soundoutput control unit is further configured to change the sound sourceposition of the sound marker according to a change in an orientation ofa face of a user.(13) The information processing apparatus of (3), wherein the soundoutput control unit is further configured to set the sound sourceposition of the sound marker as a same location as the virtual position,when the virtual position is within a visible area in front of a user.(14) The information processing apparatus of (3), wherein the soundoutput control unit is further configured to determine the sound sourceposition of the sound marker such that a user perceives output sound asoriginating from a location of the real object.(15) The information processing apparatus of (1), further including:a display unit configured to display a visual marker at the virtualposition in a superimposing manner over a video image.(16) The information processing apparatus of (1), further including:a head mounted display configured to display a visual marker at thevirtual position.(17) The information processing apparatus of (1), wherein when the realobject is present within a visible region of a user, the virtualposition is set to be located at the position of the real object.(18) The information processing apparatus of (1), wherein the soundoutput control unit is further configured to generate the localizationinformation by filtering information associated with real objectslocated in proximity to a user.(19) The information processing apparatus of (1), wherein the sound hasa directivity based on the localization information.(20) The information processing apparatus of (1), wherein a movement ofthe real object is determined by detecting a change between a firstpoint in time and a second point in time of a frequency of the sound asperceived by a user.(21) The information processing apparatus of (1), wherein the soundmarker is not visible to a user.(22) An information processing method including:generating localization information of a sound marker based on a virtualposition; andoutputting a sound associated with the sound marker, based on thelocalization information,wherein the virtual position is determined based on a position of a realobject present in a space.(23) The information processing method of (22), wherein the sound isaudibly outputted by an audio producing unit concurrently with adisplaying of a visual marker on a display unit at the virtual position.(24) The information processing method of (22), further including:displaying a visual marker at the virtual position; andselecting the visual marker as information that is desired by a user.(25) The information processing method of (24), wherein the visualmarker is selected based on a movement of a head of the user.(26) The information processing method of (22), wherein the generatingthe localization information includes determining a sound sourceposition of the sound marker.(27) The information processing method of (26), wherein the sound sourceposition of the sound marker is determined based on the virtualposition.(28) The information processing method of (26), wherein the sound sourceposition of the sound marker is determined based on a distance between aposition of a user and the virtual position.(29) The information processing method of (28), wherein when thedistance between the position of the user and the virtual position isgreater than a threshold value, the sound source position of the soundmarker is determined to be at a location that is located closer to beingdirectly in front of the user than to the virtual position.(30) The information processing method of (29), wherein when thedistance between the position of the user and the virtual position isless than a threshold value and reducing, the sound source position ofthe sound marker correspondingly moves from a front side of the usertowards the real object.(31) The information processing method of (28), wherein when thedistance between the position of the user and the virtual position isless than a threshold value, the sound source position of the soundmarker is determined to be located at a corresponding point along acircumference having a radius of the threshold value and centered on theposition of the user.(32) The information processing method of (31), wherein as the realobject passes from a front side of the user to a back side of the user,the sound source position of the sound marker moves along thecircumference from the front side of the user to the back side of theuser until reaching a position located directly behind the user at whichpoint the sound source position of the sound marker then recedes awayfrom the user along a path of a vector.(33) The information processing method of (28), wherein:when the distance between the position of the user and the virtualposition is greater than a threshold value, the sound source position ofthe sound marker is determined to be on a front side of the user; andwhen the user approaches a physical location corresponding to thevirtual position, the sound source position of the sound marker movesfrom the front side of the user in a direction towards the virtualposition.(34) The information processing method of (28), wherein:as the distance between the position of the user and the virtualposition decreases to approach a threshold value, the sound sourceposition of the sound marker moves closer to the position of the useralong a vector that is parallel to a straight ahead direction of theuser;as the distance between the position of the user and the virtualposition decreases from the threshold value to a point in time at whichthe virtual position is directly in front of the user, the sound sourceposition of the sound marker moves along a circumference having a radiusof the threshold value and centered on the position of the user towardsa position located directly in front of the user, andas the virtual position moves from being directly in front of the usertowards a back side of the user, the sound source position of the soundmarker moves from the position located directly in front of the user andalong the circumference towards the back side of the user and out of avisible region of the user.(35) The information processing method of (26), further including:changing the sound source position of the sound marker according to achange in an orientation of a face of a user.(36) The information processing method of (26), further including:setting the sound source position of the sound marker as a same locationas the virtual position, when the virtual position is within a visiblearea in front of a user.(37) The information processing method of (26), wherein the sound sourceposition of the sound marker is determined such that a user perceivesoutput sound as originating from a location of the real object.(38) The information processing method of (22), wherein a visual markeris displayed at the virtual position by a display unit in asuperimposing manner over a video image.(39) The information processing method of (22), wherein a visual markeris displayed at the virtual position by using a head mounted displaythat is worn by a user.(40) The information processing method of (22), wherein when the realobject is present within a visible region of a user, the virtualposition is set to be located at the position of the real object.(41) The information processing method of (22), wherein the generatingthe localization information includes filtering information associatedwith real objects located in proximity to a user.(42) The information processing method of (22), wherein the sound has adirectivity based on the localization information.(43) The information processing method of (22), further including:determining a movement of the real object by detecting a change betweena first point in time and a second point in time of a frequency of thesound as perceived by a user.(44) The information processing method of (22), wherein the sound markeris not visible to a user.(45) A non-transitory computer-readable medium embodied with a program,which when executed by a computer, causes the computer to perform amethod including:generating localization information of a sound marker based on a virtualposition; andoutputting a sound associated with the sound marker, based on thelocalization information,wherein the virtual position is determined based on a position of a realobject present in a space.(46) The computer-readable medium of (45), further including:displaying a visual marker at the virtual position; andselecting the visual marker as information that is desired by a user.(47) The computer-readable medium of (46), wherein the visual marker isselected based on a movement of a head of the user.(48) The computer-readable medium of (45), wherein the generating thelocalization information includes determining a sound source position ofthe sound marker.(49) The computer-readable medium of (48), wherein the sound sourceposition of the sound marker is determined based on a distance between aposition of a user and the virtual position.(50) The computer-readable medium of (49), wherein when the distancebetween the position of the user and the virtual position is greaterthan a threshold value, the sound source position of the sound marker isdetermined to be at a location that is located closer to being directlyin front of the user than to the virtual position.(51) The computer-readable medium of (50), wherein when the distancebetween the position of the user and the virtual position is less than athreshold value and reducing, the sound source position of the soundmarker correspondingly moves from a front side of the user towards thereal object.(52) The computer-readable medium of (49), wherein when the distancebetween the position of the user and the virtual position is less than athreshold value, the sound source position of the sound marker isdetermined to be located at a corresponding point along a circumferencehaving a radius of the threshold value and centered on the position ofthe user.(53) The computer-readable medium of (50), wherein as the real objectpasses from a front side of the user to a back side of the user, thesound source position of the sound marker moves along the circumferencefrom the front side of the user to the back side of the user untilreaching a position located directly behind the user at which point thesound source position of the sound marker then recedes away from theuser along a path of a vector.(54) The computer-readable medium of (49), wherein:when the distance between the position of the user and the virtualposition is greater than a threshold value, the sound source position ofthe sound marker is determined to be on a front side of the user; andwhen the user approaches a physical location corresponding to thevirtual position, the sound source position of the sound marker movesfrom the front side of the user in a direction towards the virtualposition.(55) The computer-readable medium of (49), wherein:as the distance between the position of the user and the virtualposition decreases to approach a threshold value, the sound sourceposition of the sound marker moves closer to the position of the useralong a vector that is parallel to a straight ahead direction of theuser;as the distance between the position of the user and the virtualposition decreases from the threshold value to a point in time at whichthe virtual position is directly in front of the user, the sound sourceposition of the sound marker moves along a circumference having a radiusof the threshold value and centered on the position of the user towardsa position located directly in front of the user, andas the virtual position moves from being directly in front of the usertowards a back side of the user, the sound source position of the soundmarker moves from the position located directly in front of the user andalong the circumference towards the back side of the user and out of avisible region of the user.(56) The computer-readable medium of (48), further including:changing the sound source position of the sound marker according to achange in an orientation of a face of a user.(57) An information processing device including:an acquisition unit that acquires a virtual position of a virtual objectdetermined based on a position of a real object present in a space; anda localization information generation unit that generates localizationinformation indicating a sound source position of sound informationassociated with the virtual object based on the virtual position.(58) The information processing device according to (57), wherein thelocalization information generation unit generates the localizationinformation based on the distance between a user position and thevirtual position.(59) The information processing device according to (58), wherein, whenthe distance between the user position and the virtual position is equalto or longer than a predetermined threshold value, the localizationinformation generation unit causes the sound source position to getcloser to the front side of the user than to the virtual position.(60) The information processing device according to (59), wherein, whenthe distance between the user position and the virtual position isshorter than the threshold value, the localization informationgeneration unit causes the sound source position to move from the frontside of the user to the direction of the real object.(61) The information processing device according to (60), wherein, whenthe distance between the user position and the virtual position isshorter than the threshold value, the localization informationgeneration unit causes the sound source position to move along acircumference having the threshold value as a radius and having the userposition as a center.(62) The information processing device according to any one of (57) to(61), wherein, when the virtual position is located in a predeterminedregion in the front direction of the user, the localization informationgeneration unit sets the virtual position at the sound source position.(63) The information processing device according to any one of (57) to(62), further including:an orientation detection unit that detects an orientation of a user'sface.wherein the localization information generation unit generates thelocalization information based on the orientation of the face.(64) The information processing device according to any one of (57) to(63), further including:a display control unit that causes the virtual object to be displayed atthe virtual position.(65) The information processing device according to (64), furtherincluding:a sound output control unit that causes the sound information to beoutput based on the localization information when a selection operationof selecting the displayed virtual object is detected.(66) The information processing device according to (64) or (65),wherein the display control unit causes primary information of thevirtual object to be displayed at the virtual position, and causessecondary information of the virtual object to be displayed when aselection operation of selecting the virtual object is detected.(67) The information processing device according to (65) or (66),wherein the selection operation is detected based on an orientation of auser's face.(68) An information processing method including:acquiring a virtual position of a virtual object determined based on aposition of a real object present in a space; andgenerating localization information indicating a sound source positionof sound information associated with the virtual object based on thevirtual position.(69) A program that causes a computer to function as an informationprocessing device including:an acquisition unit that acquires a virtual position of a virtual objectdetermined based on a position of a real object present in a space; anda localization information generation unit that generates localizationinformation indicating a sound source position of sound informationassociated with the virtual object based on the virtual position.

The present disclosure contains subject matter related to that disclosedin Japanese Priority Patent Application JP 2012-021888 filed in theJapan Patent Office on Feb. 3, 2012, the entire content of which ishereby incorporated by reference.

1. An information processing apparatus comprising: a sound outputcontrol unit configured to generate localization information of a soundmarker based on a virtual position; and a sound output unit configuredto output a sound associated with the sound marker, based on thelocalization information, wherein the virtual position is determinedbased on a position of a real object present in a space.
 2. Theinformation processing apparatus of claim 1, wherein the sound outputunit is further configured to audibly output the sound concurrently witha displaying of a visual marker on a display unit at the virtualposition.
 3. The information processing apparatus of claim 1, whereinthe sound output control unit is further configured to determine a soundsource position of the sound marker.
 4. The information processingapparatus of claim 3, wherein the sound output control unit is furtherconfigured to determine the sound source position of the sound markerbased on the virtual position.
 5. The information processing apparatusof claim 3, wherein the sound output control unit is further configuredto determine the sound source position of the sound marker based on adistance between a position of a user and the virtual position.
 6. Theinformation processing apparatus of claim 5, wherein when the distancebetween the position of the user and the virtual position is greaterthan a threshold value, the sound output control unit is furtherconfigured to determine the sound source position of the sound marker tobe at a location that is located closer to being directly in front ofthe user than to the virtual position.
 7. The information processingapparatus of claim 6, wherein when the distance between the position ofthe user and the virtual position is less than a threshold value andreducing, the sound output control unit is further configured todetermine the sound source position of the sound marker ascorrespondingly moving from a front side of the user towards the realobject.
 8. The information processing apparatus of claim 5, wherein whenthe distance between the position of the user and the virtual positionis less than a threshold value, the sound output control unit is furtherconfigured to determine the sound source position of the sound marker tobe located at a corresponding point along a circumference having aradius of the threshold value and centered on the position of the user.9. The information processing apparatus of claim 8, wherein as the realobject passes from a front side of the user to a hack side of the user,the sound output control unit is further configured to determine thesound source position of the sound marker as moving along thecircumference from the front side of the user to the back side of theuser until reaching a position located directly behind the user at whichpoint the sound source position of the sound marker then recedes awayfrom the user along a path of a vector.
 10. The information processingapparatus of claim 5, wherein: when the distance between the position ofthe user and the virtual position is greater than a threshold value, thesound output control unit is further configured to determine the soundsource position of the sound marker to be on a front side of the user;and when the user approaches a physical location corresponding to thevirtual position, the sound output control unit is further configured todetermine the sound source position of the sound marker as moving fromthe front side of the user in a direction towards the virtual position.11. The information processing apparatus of claim 5, wherein: as thedistance between the position of the user and the virtual positiondecreases to approach a threshold value, the sound output control unitis further configured to determine the sound source position of thesound marker as moving closer to the position of the user along a vectorthat is parallel to a straight ahead direction of the user; as thedistance between the position of the user and the virtual positiondecreases from the threshold value to a point in time at which thevirtual position is directly in front of the user, the sound outputcontrol unit is further configured to determine the sound sourceposition of the sound marker a moving along a circumference having aradius of the threshold value and centered on the position of the usertowards a position located directly in front of the user; and as thevirtual position moves from being directly in front of the user towardsa back side of the user, the sound output control unit is furtherconfigured to determine the sound source position of the sound marker asmoving from the position located directly in front of the user and alongthe circumference towards the back side of the user and out of a visibleregion of the user.
 12. The information processing apparatus of claim 5,wherein the sound output control unit is further configured to changethe sound source position of the sound marker according to a change inan orientation of a face of a user.
 13. The information processingapparatus of claim 3, wherein the sound output control unit is furtherconfigured to set the sound source position of the sound marker as asame location as the virtual position, when the virtual position iswithin a visible area in front of a user.
 14. The information processingapparatus of claim 3, wherein the sound output control unit is furtherconfigured to determine the sound source position of the sound markersuch that a user perceives output sound as originating from a locationof the real object.
 15. The information processing apparatus of claim 1,further comprising: a display unit configured to display a visual markerat the virtual position in a superimposing manner over a video image.16. The information processing apparatus of claim 1, further comprising:a head mounted display configured to display a visual marker at thevirtual position.
 17. The information processing apparatus of claim 1,wherein when the real object is present within a visible region of auser, the virtual position is set to be located at the position of thereal object.
 18. The information processing apparatus of claim 1,wherein the sound output control unit is further configured to generatethe localization information by filtering information associated withreal objects located in proximity to a user.
 19. The informationprocessing apparatus of claim 1, wherein the sound has a directivitybased on the localization information.
 20. The information processingapparatus of claim 1, wherein a movement of the real object isdetermined by detecting a change between a first point in time and asecond point in time of a frequency of the sound as perceived by a user.21. The information processing apparatus of claim 1, wherein the soundmarker is not visible to a user.
 22. An information processing methodcomprising: generating localization information of a sound marker basedon a virtual position; and outputting a sound associated with the soundmarker, based on the localization information, wherein the virtualposition is determined based on a position of a real object present in aspace.
 23. A non-transitory computer-readable medium embodied with aprogram, which when executed by a computer, causes the computer toperform a method comprising: generating localization information of asound marker based on a virtual position; and outputting a soundassociated with the sound marker, based on the localization information,wherein the virtual position is determined based on a position of a realobject present in a space.